Skip to content

Align Smoke Copilot prompts with actual tool names#40576

Merged
pelikhan merged 5 commits into
mainfrom
copilot/copilotupdate-conclusion-job-aggregate-data
Jun 21, 2026
Merged

Align Smoke Copilot prompts with actual tool names#40576
pelikhan merged 5 commits into
mainfrom
copilot/copilotupdate-conclusion-job-aggregate-data

Conversation

Copilot AI commented Jun 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Smoke Copilot was reporting missing tools even though equivalent capabilities were available in the workflow environment. The issue was in the prompt surface: it asked for generic/nonexistent tool names instead of the concrete tools exposed to Copilot.

  • Prompt/tool-name alignment

    • Updated the Smoke Copilot workflow prompts to reference the actual tools the runtime exposes:
      • github (configured with mode: gh-proxy) instead of github-mcp-server
      • web-fetch instead of a generic “web fetch tool”

  • Applied consistently across Copilot smoke variants

    • Updated:
      • smoke-copilot.md
      • smoke-copilot-aoai-apikey.md
      • smoke-copilot-aoai-entra.md
    • Recompiled the corresponding .lock.yml files so generated workflows reflect the prompt changes.

  • Effect

    • Keeps the smoke checks exercising the intended capabilities while avoiding false missing-tool reports in successful runs.
- Before: GitHub MCP: review 2 merged PRs in `${{ github.repository }}`
- After:  `github` tool (configured with `mode: gh-proxy`): review 2 merged PRs in `${{ github.repository }}`

- Before: Web fetch tool: fetch `https://github.com` and confirm response contains `GitHub`
- After:  `web-fetch` tool: fetch `https://github.com` and confirm response contains `GitHub`

Copilot AI and others added 2 commits June 21, 2026 04:56
@pelikhan
chore: start investigating smoke-copilot missing tools
6ede44d 
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
fix: align smoke copilot prompts with available tools
e0f9b04 
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix missing tools in Smoke Copilot workflow Align Smoke Copilot prompts with actual tool names Jun 21, 2026
Copilot AI requested a review from pelikhan June 21, 2026 05:06
@pelikhan pelikhan marked this pull request as ready for review June 21, 2026 05:07
Copilot AI review requested due to automatic review settings June 21, 2026 05:07
@pelikhan

pelikhan commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator

@copilot merge main and recompile

Copilot AI reviewed Jun 21, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Smoke Copilot workflow prompts to reference the concrete tool names actually exposed in the runtime environment, then recompiles the generated .lock.yml workflows so the compiled outputs reflect the prompt changes.

Changes:

  • Replaced generic/nonexistent prompt references (“GitHub MCP”, “Web fetch tool”) with the actual tool names (github with mode: gh-proxy, and web-fetch).
  • Applied the prompt updates across the Smoke Copilot variants (base, AOAI apikey, AOAI Entra).
  • Recompiled the corresponding .lock.yml files (metadata hashes updated).
Show a summary per file
File Description
.github/workflows/smoke-copilot.md Updates the test checklist to reference github (gh-proxy) and web-fetch explicitly.
.github/workflows/smoke-copilot.lock.yml Recompiled output reflecting prompt body changes (metadata hash update).
.github/workflows/smoke-copilot-aoai-apikey.md Updates the test checklist to reference github (gh-proxy) and web-fetch explicitly.
.github/workflows/smoke-copilot-aoai-apikey.lock.yml Recompiled output reflecting prompt body changes (metadata hash update).
.github/workflows/smoke-copilot-aoai-entra.md Updates the test checklist to reference github (gh-proxy) and web-fetch explicitly.
.github/workflows/smoke-copilot-aoai-entra.lock.yml Recompiled output reflecting prompt body changes (metadata hash update).

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 6/6 changed files
  • Comments generated: 2

Comment thread .github/workflows/smoke-copilot-aoai-apikey.md
Run each check NOW and mark as ✅/❌. Do NOT create files to automate this — execute directly using bash and tools:

1. GitHub MCP: review 2 merged PRs in `${{ github.repository }}`.
1. `github` tool (configured with `mode: gh-proxy`): review 2 merged PRs in `${{ github.repository }}`.
Comment thread .github/workflows/smoke-copilot-aoai-entra.md
Run each check NOW and mark as ✅/❌. Do NOT create files to automate this — execute directly using bash and tools:

1. GitHub MCP: review 2 merged PRs in `${{ github.repository }}`.
1. `github` tool (configured with `mode: gh-proxy`): review 2 merged PRs in `${{ github.repository }}`.
Copilot AI and others added 2 commits June 21, 2026 05:18
@pelikhan
Merge main; fix remaining 'GitHub MCP' references in EXECUTE IMMEDIAT…
145440b 
…ELY prompts

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
chore: merge main and recompile; fix remaining GitHub MCP refs in EXE…
45d88cc 
…CUTE IMMEDIATELY prompts

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 21, 2026

Copy link
Copy Markdown
Contributor Author

@copilot merge main and recompile

Done. Merged main (resolved a single body_hash conflict in the generated lock file), fixed the remaining GitHub MCP references in the EXECUTE IMMEDIATELY lines of smoke-copilot-aoai-apikey.md and smoke-copilot-aoai-entra.md (now reference `github` tool), and recompiled all 249 workflows — commit chore: merge main and recompile; fix remaining GitHub MCP refs in EXECUTE IMMEDIATELY prompts.

@github-actions

github-actions Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

@copilot please confirm whether anything remains before merge, or summarize the final blocker if any.

Generated by 👨‍🍳 PR Sous Chef · 55.7 AIC · ⌖ 1 AIC · ⊞ 17.3K · ◷

This was referenced Jun 21, 2026
Closed
Closed
@github-actions

github-actions Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

Please summarize the remaining blockers and the next commit needed to close this out.

Generated by 👨‍🍳 PR Sous Chef · 61 AIC · ⌖ 1.03 AIC · ⊞ 17.3K · ◷

@github-actions

github-actions Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

@copilot review all comments and address unresolved review feedback about replacing “GitHub MCP” with the github tool.

Generated by 👨‍🍳 PR Sous Chef · 61 AIC · ⌖ 1.03 AIC · ⊞ 17.3K · ◷

@github-actions github-actions Bot mentioned this pull request Jun 21, 2026
Closed
@pelikhan pelikhan merged commit 8e935e3 into main Jun 21, 2026
25 checks passed
@pelikhan pelikhan deleted the copilot/copilotupdate-conclusion-job-aggregate-data branch June 21, 2026 13:01
This was referenced Jun 21, 2026
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@pelikhan pelikhan Awaiting requested review from pelikhan

Assignees

@pelikhan pelikhan

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@pelikhan